By Michael Lagally
Curious about how to write your own 3D Java applications for your home theater? Learn how from this article, which explains Blu-ray 3D concepts and the BD-J platform.
Imagine on your TV the big letters “Hello World” flashing in 3D. Well, there are certainly many more interesting applications you can imagine on a TV, but almost every programming experience starts with “Hello World.” This article gives you the concepts and the background information you need to create your first 3D Hello World application in Java.
First you need a Java platform connected to your TV. The good news is that you most likely already have a full-blown Java platform connected to your TV in the form of a Blu-ray player. The latest Blu-ray players are capable of displaying 3D content, and since the Blu-ray standard comes with Java 3D APIs, you are good to go.
Even if you don’t have a 3D TV or a Blu-ray player, you can still start developing and testing 3D programs. All you need, provided you own a Microsoft Windows computer, is free software and a pair of red/blue glasses (also called anaglyph glasses) that you can get for a few dollars.
Before you can get down to the actual coding, you need to understand a bit about the Java platform for Blu-ray.
The platform for Blu-ray is called BD-J. BD-J is based on the Java Platform, Micro Edition (Java ME) Personal Basic Profile (PBP) of the Connected Device Configuration (CDC) and the Java TV specification. It is built on standardized DVB-GEM (Digital Video Broadcasting-Globally Executable Multimedia Home Platform) middleware, which is also the foundation for other TV middleware standards, such as tru2way for US cable. The BD-J stack combines the well-known benefits of the Java ME run-time environment with additional UI, media, and disc handling APIs.
You have a full Java platform in your player that includes a network stack with Secure Sockets Layer (SSL) and a multi-application run-time environment that can concurrently run multiple Xlets and offers inter-Xlet communication, a resource management framework, and more.
BD-J applications are just Xlets that can be synchronized with a Blu-ray movie and have access to some new APIs.
The BD-J platform was recently extended with some 3D APIs that allow you to handle 3D displays and draw to separate frame buffers for the left and right eyes. The new 3D Profile (Profile 5) is basically just the normal profile of a connected player (Profile 2) plus the following stereoscopic 3D extensions:
A new movie codec for frame-sequential 3D (MPEG-4 MVC)
Metadata extensions to enable the signaling of 2D/3D titles
Several BD-J API extensions:
The next sections present the BD-J API extensions in more detail.
The 3D modes in Blu-ray are an extension to the drawing APIs, which offer a (limited) way to draw 3D scenes. The capabilities can be compared to a theater stage, where flat backgrounds are set at different depths to give a 3D perception.
There’s an API to query the 3D capabilities of a player and the display that’s attached to it. This API also is used to switch between the different 3D modes and back to 2D.
The 3D APIs for BD-J support two different 3D display modes. One is called “1-plane + offset,” whereas the other is called two-plane mode.
As shown in Figures 1 and 2, the “1 plane + offset” mode basically gives you a single drawing plane, on which you can use all draw APIs as you would do in a non-3D application. The API gives you a method to set an offset between the display for the left and right eyes. This offset is applied when the plane is displayed for the left and right eyes and makes the plane appear closer to you or further away, depending on whether the offset has a positive or negative value.
Figure 1: “1-plane+offset” Mode
Since the offset is always applied to the full plane, all draw objects appear at the same depth; it is not possible to put individual items at different depths. Because there’s only one frame buffer, which gets shifted, there’s no need to synchronize any draw operations, so you can use all the Abstract Window Toolkit (AWT)-based draw operations that you were using for a 2D BD-J application. It is, therefore, relatively easy to adapt an existing BD-J application to use the “1 plane + offset” mode.
Figure 2: Drawing Model in the "1 Plane + Offset" Mode
As shown in Figures 3 and 4, the two-plane mode gives you two full-size screen buffers, one for the left eye and one for the right eye. The normal drawing API does not work in two-plane mode, since synchronized drawing to two frame buffers cannot be guaranteed.
Instead, there’s a new drawing API that provides methods to do synchronized blitting to the left and right screen pane. The application draws to an off-screen buffer and uses the drawStereoscopic API to update the left and right frame buffers simultaneously. This API enables the placement of objects at different perceived depths, so the two-plane mode can be used to create a more sophisticated 3D scene. It’s worth noting that you still use the full set of AWT draw operations to render to the off-screen buffers.
Figure 3: Two-Plane Mode
The two-plane mode offers you the highest flexibility due to individual depth placement of objects and is suitable for advanced menus, animations, games, and so on. But beware: This mode is still handling only 2D shapes and is not a full volumetric 3D API in the manner of OpenGL (JSR 239).
Figure 4: Drawing Model in Two-Plane Mode
The API that is used for stereoscopic drawing in two-plane mode offers two draw methods. In the simple form, it just takes two images and two arrays of update rectangles. When you call it, the API guarantees that at least the contents of the update areas are copied into your frame buffers for the left eye and right eye. You can help your player minimize the redraw effort by specifying the update rectangles. If you want to copy the full images, you just use NULL.
public void drawStereoscopic( java.awt.Image left, int leftX, int leftY, java.awt.Rectangle leftUpdateAreas, java.awt.Image right, int rightX, int rightY, java.awt.Rectangle rightUpdateAreas);
An extended form of the drawStereoscopic API, drawStereoscopicImages, works on arrays of images to be simultaneously copied.
public void drawStereoscopicImages( java.awt.Image leftImages, int leftXs, int leftYs, java.awt.Rectangle leftUpdateAreas, java.awt.Image rightImages, int rightXs, int rightYs, java.awt.Rectangle rightUpdateAreas);
For each image, there’s a one-to-one mapping between the images and the coordinates. When update areas are specified, the API guarantees to copy only the content of the update areas. NULL can be used for the update areas to copy the complete images.
Note that since both APIs don’t modify the source images, the left and right image instances could be the same image objects.
In a 3D world, where you blend a graphics layer with a 3D movie (video) layer, you have to solve a problem: What happens if you put your graphics layer at a certain depth and the movie content comes in front of it? You certainly have a bad visual appearance, as illustrated in Figure 5.
In the left picture, everything is fine and the graphics layer is nicely in front of the video layer. But in the right picture, the video layer comes closer and the red areas overlap with the graphics layer.
You always need to ensure that your menus and subtitles are presented in front of the video.
Figure 5: Offset Metadata
A Blu-ray player can adjust the position of the graphics layer automatically from a sequence of offset metadata, which typically is created during the authoring process of the movie. This offset metadata indicates the shift of the graphics layer at each time position of the movie. This information allows dynamic adjustment of the depth position of the graphics plane as the 3D scene of the movie changes depth. There may be more than one offset sequence for an audio-video (A/V) stream to accommodate different viewer preferences; there could also be titles without offset metadata.
The Java API for offset metadata allows you to select an offset sequence or query the currently selected sequence number.
GraphicsOffsetSequenceControl is a Java Media Framework (JMF) control to get or set the sequence ID of the offset sequence for the JMF player. It provides a change listener mechanism, which is notified about changes in the offset sequence ID.
The offset values can change at each frame. The API handles only the offset sequences, not the individual offset value. There’s no programmatic way to determine the actual offset shift, since the actual shifting of the graphics layer is happening inside the player engine without application involvement. Offset metadata can be used for the ”1-plane + offset” mode and the two-plane mode.
The 3D BD-J extensions offer several other small APIs for doing the following:
Selecting the alignment of subtitles (top, bottom)
Querying the 3D capabilities of the display
Now that you have read this article, you have an idea of the 3D concepts for BD-J. If you now are eager to start your own development, you need to set up your development environment for BD-J, get a couple of free tools, and get a software player.
You can create your applications and your Blu-ray disk images on your hard disk and mount them as virtual disks into your software Blu-ray player. Several software players can render to anaglyph, which is a red and blue encoding of the 3D information on a normal display. It’s a compromise, but it is the cheapest way to have a 3D perception without the need to buy shutter glasses and 3D-capable displays. I have successfully done some demo applications on a state-of-the-art laptop without any additional hardware.
Due to the fact that several pieces from different standards are combined into the BD-J platform, there is no single source for a complete set of JavaDocs for the platform. The documentation for the different layers of the BD-J stack is available from the following sources:
Free-of-charge license for BD-J JavaDocStubs 2D/3D from the Blu-ray Disc Association License Office: http://blu-raydisc.info/license-app/javadocstubs-app.php
Note: This package contains the API documentation for the BD-J APIs, including the 3D APIs that were explained in this article.
GEM Stubs for Packaged Media Target (GEM 1.0.3): http://www.mhp.org/mhpgem10.htm
Here are some additional resources:
The technical article “Building Javadoc API Reference Documentation for Blu-ray Disc Application Development” describes how to combine the individual JavaDocs into a single JavaDoc for the full BD-J platform: http://www.oracle.com/technetwork/java/javame/bluray-technote-142391.html
The official BD-ROM white paper gives a good overview of the format: http://www.blu-raydisc.com/assets/Downloadablefile/BD-ROM-AV-WhitePaper_100604%281%29-15916.pdf
The HD cookbook open-source project includes frameworks and tools for BD-J and other GEM platforms for creating Blu-ray disc applications: http://java.net/projects/hdcookbook
About the Author
|Michael Lagally is a principal member of technical staff at Oracle. He works in the
technology management group and represents Oracle in TV-related and other
standardization organizations, such as the DVB, the Blu-ray Disc Association,
and the Java Community Process (JCP).