As Published In May/June 2006

Making Movie Magic
By Thomas Ulrich

Lucasfilm expands its empire with technology.

When people sit in a darkened movie theater or play a video game, they seldom think about the technical requirements for animation or special effects that the producer needed to bring the story to life. Technology plays a bigger role today than it ever did in entertainment, and the role of technology in digital entertainment—including digital film and electronic games—isn't just big. It's a blockbuster.

George Lucas is the creator of the Star Wars franchise among other box-office smashes, and Oracle is one technology company that has helped him realize his artistic vision. Lucas' company, Lucasfilm, recently moved its headquarters to the Letterman Digital Arts Center, located in San Francisco's Presidio National Park, and consolidated LucasArts (electronic games) and Industrial Light & Magic (ILM; special effects) in the new complex.

The convergence of these entertainment divisions onto one campus represents more than just convenient geography—it represents the increasingly blurred technical boundaries of different types of digital entertainment. For example, LucasArts artists can use lighting techniques in games that ILM artists designed for film, and engineers can let directors preview a special effect staged for the cinema from a handheld computer. Oracle, along with key business partners, has accelerated this trend by providing the tools and services that make it possible.

As Lucasfilm's entertainment divisions cross-pollinate, an Oracle database enables the creative force to flourish. Let's step behind the curtain and see how Oracle's technical capacity can expand the horizon of creative vision.

The Power Within

To design, manipulate, access, and store the vast number of complex images that go into a creative work, artists at Lucasfilm need huge amounts of computing capacity. The digital arts center is outfitted with a database from Oracle, routers from Foundry Networks, and workstations from HP to deliver content to about 1,500 artists over a broadband network.

The digital arts center has the largest computer network in the entertainment industry, with more than 3,000 processors that render complex special effects or images and provide network services such as e-mail. More than 1,500 workstations are connected to an Oracle database, so artists do not have to wait for the dailies to review the latest cut. They can e-mail work in progress to a visual effects supervisor for immediate review and watch high-definition clips of a recently composed scene without leaving their desks. In addition, they can rework their part of the story independently by managing large data files containing all the elements of a scene.

"Oracle underlies hundreds of proprietary scripts and tools," says Jeff Odell, Lucasfilm's Oracle database administrator. "Data is such a central component of our pipeline and is used by employees with such diverse roles that almost all of our internal tools touch the database. From single-purpose command-line scripts to Oracle hooks in our computer graphics software, the database is everywhere."

Starting from the Top

Every production at Lucasfilm begins with design, and every image begins at the database. Before artists craft an image, the database provides live-action frames, stock footage, motion picture capture data, and metadata to help them manage a project. Once a director has laid out a storyboard—an illustrated outline of the work—an artist creates a low-resolution version of, say, a synthetic character. When the director approves the concept, the artist builds a geometric framework or skeleton from a web of polygons; adds texture, color, and perhaps movement; and then composes the elements into a sequence that the director places in a scene.

An animated sequence requires the artist to create a model of an element by describing it mathematically and then bring it to life by changing its position within the frame over time. Network computers then render the image by determining the position of an element relative to other elements that appear in the frame. Finally, the artist checks the color of a finished image so that all the elements of a game sequence or special effect appear at the same time and fit neatly into the frame.

"The work of each artist passes through a complex review process involving supervisors as well as outside clients," Odell says. "The construction of the digital model, for example, takes place early in the project, but for specific shots, many refinements will probably occur through the completion of the show. All these changes are tracked and managed in the Oracle database as the work passes between artists, supervisors, and clients."

The database follows every frame of film and game sequence, from the time an artist creates an image until it is stored on backup tape. It receives 20 million database requests a day.

"The reliability and stability of our Oracle database motivated the last few holdouts at ILM to move from other database technologies into Oracle," Odell explains. "For the volume of transactions we have hitting the database, Oracle's clean management of concurrency and consistency has been crucial."

"Oracle database is the standard around which we've built the rest of the network," says Lori Gianino, Lucasfilm's director of Information Systems.

The Heart of the Empire

Database reliability is key for Lucasfilm. The database must always be available. If it failed, artists could not retrieve assets, rework scenes, or schedule network computers to render images. Production on very expensive projects would grind to a halt.

"Even with our whole production pipeline dependent on a database, we run with a very safe margin of performance overhead on our servers," Odell says. "This is particularly useful because the cyclical nature of film production means we sometimes need to manage huge spikes in activity."

The database handles peak flow by serving artists during the day and rendering complex images at night. "The load on our database remains evenly distributed throughout the day," Odell says. "During work hours, artists use the database to check work in and out. At night, the volume for rendering images increases dramatically, which includes highly granular tracking of render jobs through the database. We manage the virtual file systems on our production SAN [storage area network] in Oracle. This lets us more-or-less dynamically balance render loads across different file server heads without interrupting running jobs."

Rendering has expensive computational overhead because elements such as fire, water, or even clouds are drawn without surfaces to map and require attributes such as direction, speed, color, and transparency. To manage this load, proprietary software tools allow the data center to add workstations to the pool of computers rendering images or special effects after regular business hours, expanding capacity to 5,000 processors. The database monitors and logs each rendered image to identify and manage errors and creates statistical data so that the network can manage future project schedules.

Snapshot Lucasfilm www.lucasfilm.com Location: San Francisco, California Size: US$15 billion Oracle Products: Oracle Database

When a film or game sequence earns final approval, that version of the shot is assigned a metadata flag indicating final status. Its rendered frames and metadata are forwarded to a film-printing facility. The final sequence, source material, and metadata are stored to tape. An Oracle database manages and maintains the archives.

Preserving the Legacy

Maintaining these archives is a huge task. "Ultimately, we are charged with managing storage of all the assets for a production that may require several years to produce," says Cliff Plumer, Lucasfilm's chief technology officer.

The database stores either the asset itself or information describing where it resides. Assets that a LucasArts or ILM artist creates but another company owns are readily available, should the customer request them. The special effects that an artist creates for Lucasfilm can be stored in an archive once the film crew completes production.

"ILM provides special effects for several feature films each year," Odell says. "These effects can range from alien armies and historical cityscapes to touching up actors' blemishes. Between the hugely different needs of these projects and ILM's continuing technical advances in special effects, the data we maintain for each film can vary significantly. To accommodate these projects in a reasonably static pipeline and to ensure that we'll be able to revisit any customized data easily when sequels roll around, integration is a central concern."

Next Steps LEARN more about Oracle Database 10g DOWNLOAD Oracle Database 10g

Odell says that the architecture helps them manage data volume and exchanges. "The heart of our approach is translating and isolating a core set of data that's uniform across shows," he explains. "More-exotic data can then be layered on top of central tables and schemas. This also helps us manage the large volumes of data we exchange with movie studios and film scanning houses. The solid architecture we've established for integrating our own changing internal data helps simplify these large data exchanges."

In addition to providing artists with the electronic tools they need to create and store images, Lucasfilm uses the database for managing assets, workflow, production, and licenses.

"With so many technical people working at the Lucas companies and so much technological innovation going into every project, tools for accessing our data are replaced or evolve much more quickly than in other industries," Odell says. "This requires careful monitoring, constant tuning, and a durable database. Every shot that leaves ILM generates thousands, if not tens of thousands, of rows in our databases as it passes through our asset and render management pipelines."

Coming Attractions: Singapore

Preserving past productions and maintaining the Lucasfilm legacy offers one kind of technical challenge, and looking forward presents another. The company plans to open a Singapore animation studio, and this division, while broadening the entertainment giant's global reach, will present unique integration and other technical issues.

"We look forward to sharing data with Lucasfilm Animation Singapore, a digital studio designed to create movies and television programs for a global audience," says Odell. "We are considering Oracle Database 10g because it offers us some interesting options, including time zone management."

Integrating the work of the Singapore operation into the Letterman Digital Arts Center will be tricky. The time difference could be a concern for database load, because its working hours correspond almost exactly to the heaviest rendering hours at the San Francisco complex.

"For our developers, I think Oracle Data Pump and time zone management are the biggest draws," Odell says. "I'm excited about [Oracle Database] 10g for the expanded automation of DBA tasks."

Technical improvements alone can't fire the imagination. Art design, plot, and character development depend on vision. Still, creative ideas can be expanded and enhanced—and their global reach extended—by the IT forces in San Francisco or Singapore. Or even in a galaxy far, far away.