As Published In July/August 2003

Multimedia and Geospatial Data

By Joseph Mauro

Work with rich data in J2EE applications using Oracle9i Database, Oracle9iAS, and interMedia.

With colorful animations and maps springing to life nowadays on everything from ATM machines to automobiles, it's clear that applications that can take advantage of multimedia and geospatial data are increasingly in demand. Application developers eager to gain a foothold in the use of this type of rich visual data face several technical hurdles. These include how to get large media objects from data stores; how to process, render, and visualize such data; and how to make the solutions portable and scalable. The Java 2 Platform, Enterprise Edition (J2EE), offers several application server technologies that simplify the development and deployment of this kind of application. These technologies include Java Classes for Servlets and JavaServer Pages (JSP), JSP Tag Libraries, JSP Data Tag Libraries for Java integrated development environments (IDEs), Image Adaptation Servlets based on Sun's Java Advanced Imaging (JAI) API, Map Visualization Servlets, and Java Database Connectivity (JDBC) to media and geospatial content. All of these J2EE technologies and components are available in Oracle9i Application Server (Oracle9iAS), Oracle Database, and development tools.

Challenges of Multimedia and Spatial Data

The first challenge middle-tier logic faces is how to address and get such data, associated metadata, and related business information from content stores such as databases. This problem is compounded by the fact that the associated business information and the media are often stored separately, meaning applications must have specialized code that understands the relationship between the media and the affiliated business information and the different techniques for accessing them.

Another problem is that of accessing a specific metadata format. This data is often bound up in the media interchange format. With a bewildering array of media formats such as BMP, JPG, WAV, and MPG, applications are faced with the task of knowing how to parse each to extract the metadata they need. As the number and type of formats grow, so do the problems the application faces. In addition to getting raw data from the content store, the application faces the problem of knowing how to manipulate such data. Handling each of the formats separately is not a viable solution. Instead, the content must be typed and bound to middle-tier objects that understand the semantics of the data.

Since multimedia and spatial datatypes are visual in nature, the application is also faced with rendering the data. Rendering is the process of preparing or formatting the data for display purposes. Rendering is almost always required, because the size, color depth, and resolution of the data rarely match the characteristics of the output display device. And although rendering multimedia can be challenging, it is somewhat more straightforward than rendering spatial data.

Much multimedia data consists of raster graphics images, or bitmaps, which are data files or structures made up of a generally rectangular array of pixels, or points of color. Spatial data is mainly made up of vector graphics that use computational geometry to represent shapes and images and visualize them on a computer monitor and other output devices. Whether the images are raster- or vector-based, the rendering problem is a particular challenge in the emerging wireless space, with its various device-specific formats, sizes, and resolutions.

Oracle interMedia

In the past, rich media was stored in flat files, because traditional databases were not equipped to easily deal with these more-complex datatypes. Databases have evolved to handle complex data, and Oracle9i meets these new rich media and Java language requirements via Oracle interMedia, an array of services for developing and deploying traditional, Web, and wireless applications that include rich media.

Through the use of media datatypes, Oracle interMedia adds support that enables Oracle9i to manage and deliver image, audio, video, and geographical location data in an integrated fashion with other enterprise information. interMedia provides the means for adding audio, image, and video columns or objects to existing database tables, inserting and retrieving multimedia data, performing image processing on a number of popular image formats, and doing limited conversion between image formats. interMedia does this by storing media metadata in the database under Oracle interMedia control. Whether media data is stored within or outside the database, interMedia manages metadata for all the media types and can automatically extract it for audio, image, and video. interMedia makes use of object types, which are similar to Java or C++ classes, to describe multimedia data. The OrdAudio, OrdImage, and OrdVideo object types have attributes and methods associated with them and can carry Multipurpose Internet Mail Extensions (MIME)-type information (such as text, images, audio, video, and other application-specific data), making them invaluable in downstream processing and rendering.

JDBC Data Access

In supporting multimedia and geospatial content, the JDBC standard for communication between a Java application and a relational database goes beyond simply providing connectivity to the database. JDBC is able to support the return of result sets that contain media and spatial data along with metadata and traditional relational business data. JDBC also has the ability to bind these datatypes to specialized middle-tier Java classes, since it is these classes that must contain much of the media-specific logic for rendering.

Oracle interMedia provides a set of Java classes (OrdImage, OrdAudio, OrdVideo) that allow applications to access and manipulate multimedia data stored in an Oracle9i database. These Java classes allow JDBC result sets to include both traditional relational data and interMedia objects so that applications can easily select and operate on a result set containing interMedia columns and other relational data. Through the use of interMedia Java classes, applications can access object attributes and invoke object methods. After a connection to the database is established, the application can query a table and get a result set of one or more rows that have media columns. Here is some code to show how to establish a JDBC connection:

public class ImageDemo
{
private String connectString = 
"jdbc:oracle:oci8:@";
private String username = "scott";
private String password = "tiger";

private Connection conn = null;
...
private void getConnection() throws 
SQLException
{
DriverManager.registerDriver(
new oracle.jdbc.driver.OracleDriver());
conn = DriverManager.getConnection(
connectString,username,password);
}

The following code begins by querying the database and getting a result set back. The call to the getCustomDatum method retrieves the OrdImage database object from the result set and loads it into the OrdImage Java object for processing by the application.

OracleResultSet rs = (OracleResultSet)stmt1
.executeQuery("SELECT image1 
FROM imgdemo 
WHERE id = 1 FOR UPDATE");
if (rs.next())
{
OrdImage imageObj = (OrdImage)rs
.getCustomDatum(1, OrdImage.getFactory());

	

Upload and Download

The flexibility of the J2EE platform gives you several options when developing J2EE media applications. For those who wish to develop in Java, Java Classes for Servlets and JSP provide media upload and retrieval support. For those wanting to develop in HTML using JSP, there are JSP Data Tag Libraries designed for specific IDEs, generic JSP Tag Libraries, and Java Classes for Servlets and JSP that offer a number of alternatives requiring differing levels of customization and coding.

Java Classes for Servlets and JSP. Oracle interMedia includes a Java component, called the Oracle interMedia Java Classes for Java Servlets and JavaServer Pages, that facilitates the uploading and retrieval of multimedia data in a Web-based environment.

The OrdHttpResponseHandler class facilitates the retrieval of media data from an Oracle database and its delivery to a browser or other HTTP client from a Java servlet. The OrdHttpJspResponseHandler class provides the same features for JSP.

The following example shows how to use the OrdHttpResponseHandler class to retrieve an image object from a database and deliver it to a browser:

OraclePreparedStatement stmt = (OraclePreparedStatement)
conn.prepareStatement("select media from photos where id = ?");
stmt.setString(1,
request.getParameter("id"));
OracleResultSet rset = (OracleResultSet)

stmt.executeQuery( );
if (rset.next( ))
{
OrdImage media = (OrdImage)rset
.getCustomDatum(1, OrdImage.getFactory( ));
OrdHttpResponseHandler handler = new OrdHttpResponseHandler (request, response);
handler.sendImage(media);
return;
}
else{
response.setStatus(
response.SC_NOT_FOUND);
}

Similarly, audio clip objects can be retrieved and sent to a browser by using the OrdAudio.getFactory and the sendAudio methods, and video clip objects can be retrieved and sent to a browser by using the OrdVideo.getFactory and the sendVideo methods. Related methods exist for getting and sending image, audio, and video data to the browser from database BLOBs and BFILEs (operating system files).

The OrdHttpJspResponseHandler class facilitates the retrieval of media data from an Oracle database and its delivery to a browser or another HTTP client from a JSP. Listing 1 shows the use of the OrdHttpJspResponseHandler within a JSP.

The same code can handle audio or video by substituting the sendAudio method or the sendVideo method for the sendImage method. Likewise, methods exist to deliver media content from BLOBs and BFILEs.

Uploading a file using an HTML form is done by encoding both the form data and the uploaded file in a POST request, using a specific multipart/form-data format. The OrdHttpUploadFormData class facilitates the processing of such requests by parsing the POST data and making the contents of regular form fields and uploaded files readily accessible to a Java servlet or a JSP. The OrdHttpUploadFormData class provides methods to access text-based form field parameters that are identical to the getParameter( ), getParameterValues( ), and getParameterNames( ) methods provided by the ServletRequest class. Access to uploaded files is provided by similar methods, namely getFileParameter( ), getFileParameterValues( ), and getFileParameterNames( ). The OrdHttpUploadFile objects returned by the getFileParameter( ) and getFileParameterValues( ) methods provide access to the MIME type, length, and contents of each uploaded file.

Listing 2 shows how to use the OrdHttpUploadFormData class. Again, similar methods exist for both audio and video.

JSP Tag Libraries. The purpose of the JSP Tag Library is to simplify the creation of JSP Web applications that support the upload and retrieval of multimedia data stored using the interMedia object types. These tags greatly simplify the process of writing multimedia JSP Web applications. Application developers no longer have to write code to construct multimedia HTML tags or write runtime components that deliver the multimedia data. Access to data in HTML file upload forms is also greatly simplified.

To illustrate how the interMedia tag library works, the following code extract illustrates the generation of an HTML <IMG> tag. The code is entirely Java, rather than HTML, with expression elements to obtain the height and width attributes. This allows it to handle the situation where the image's height and width attributes are not known.

<%
out.print( 
"<IMG SRC=\"PhotoAlbumMediaViewer.jsp" +
"?media=image&id=" + 

album.getId() + "\"" );
if ( album.getImage().getHeight() > 0 
&& album.getImage().getWidth() > 0 )
{
out.print( "height=\"" +
album.getImage().getHeight() + "\"" );
out.print( "width=\"" +
album.getImage().getWidth() + "\"" );
}
out.print( "BORDER=\"1\">" );
%>

Using the interMedia tag library, this portion of the JSP page would look like the following:

<ord:embedImage connCache = album.getConnectionCache()

table = "PHOTOS" column = "IMAGE" 
key = "<%= album.getId() %>"
image = "<%= album.getImage() %>" border="1" />

JSP Data Tag Libraries for IDEs. Because hand-coding servlets or JSP can be tedious and error-prone, many developers are turning to the use of IDEs. Oracle9i JDeveloper is a visual component-based J2EE development environment tightly integrated with Oracle9i Database and Oracle9iAS. With Oracle9i JDeveloper, it is easy to create thick and thin Java clients, servlets, JSP, EJB, and Java Beans.

A distinguishing feature of JDeveloper is Oracle Business Components for Java (BC4J), an application component framework that gives developers a set of intelligent software building blocks to manage common facilities.

The BC4J framework automatically recognizes the interMedia object types and integrates seamlessly with interMedia at different levels, thus providing developers great flexibility in creating new applications. For example, developers can choose to create JSP applications with prepackaged database manipulation components using the Business Components JSP Application Wizard. To create more-customized JSP applications, developers can use the BC4J JSP Data Tag Library.

The Business Components JSP Application Wizard allows users to generate, without programming, a JSP application that is capable of querying, inserting, updating, and deleting data from the database. The wizard automatically integrates interMedia data objects with standard relational data using BC4J JSP Data Tags for data accessing and content rendering. Users of the JSP application are able to view, insert, update, and delete multimedia content just like other textual data. Developers can debug or run the JSP application from inside JDeveloper. When the development work is done, the developers can deploy the JSP application to an application server in the form of a Web Archive (WAR) or Enterprise Archive (EAR) file from JDeveloper.

Developers can use the BC4J Data Tag Library to create JSP applications with highly customized user interfaces and application-specific database access functionality. The BC4J Data Tag library includes a set of BC4J interMedia Data Tags that make it easier to write JSP code to upload, retrieve, or render media—for example, uploading a video clip to the database or displaying an image taken from a database table. The BC4J Data Tag Library allows developers to create database-bound, multimedia-intensive JSP applications with little effort. BC4J has the following tags:

• AnchorMedia. Inserts an HTML Anchor tag for an interMedia object to a page
• EmbedAudio. Inserts an HTML OBJECT tag for an interMedia ORDAudio object to a page
• EmbedImage. Inserts an HTML IMAGE tag for an interMedia ORDImage object to a page
• EmbedVideo. Inserts an HTML OBJECT tag for an interMedia ORDVideo object to a page
• FileUploadForm. Inserts an HTML FORM tag for file upload
• MediaURL. Inserts a URL string for an interMedia object to a page

Each tag takes a list of parameters, so for example, here's how you would use EmbedImage in a JSP application, and the resulting output:

<jbo:EmbedImage
datasource="datasourceInstanceName"
mediaattr="mediaAttributeName"
[[ rowkey="rowKeyString" ] | 
[ whereclause="whereClause" ]]

[ retrievepath="customRetrievePath" ]
[ width="imageWidth" ]
[ height="imageHeight" ]
[ border="borderPixel" ]
[ align="alignOption" ]
[[ alt="alternateText" ] | 
[ altattr="alternateAttributeName" ]]
[ longdesc="longDescription" ]
/>

Example 1:

<jbo:EmbedImage datasource="ds" 
mediaattr="Pic" whereclause="id = 2" alt="family reunion picture" />

HTML Output:

<IMG src="/technology/jdeveloper-preview/help/
topic?inOHW=false&linkHelp=false&file=jar:file:/C:/helponline/oc4j/j2ee/home/
applications/HelpOnlineApp/jdeveloper-preview/helpsets/jdeveloper-preview/
webapps/creating_html_clients.zip!/
[mediaFetchingURL]" ALT="family reunion picture" WIDTH="400" HEIGHT="350">

Wireless Rendering

The emergence of a new wave of PDAs, cell phones, and other wireless devices with larger and deeper displays has created an opportunity to deliver raster graphics (bitmap) images to them. But other than being able to display raster graphics, these devices do not follow any standards when it comes to display size, depth, number of colors, or media format.

One way to address this problem is to require content managers to replicate individual pieces of content (in the necessary device format) for every device to which they wish to output, which is a costly process storagewise. Another solution is to format or "adapt" the content to the target device on the fly. Oracle9iAS Wireless is a wireless application server that enables this kind of device-independent application development. Oracle9iAS Wireless uses the Oracle9iAS Image Adaptation Servlet to dynamically transcode images and adapt them to suit different client devices.

The Image Adaptation Servlet is written in Java and runs inside the Oracle Containers for J2EE (OC4J). The Image Adaptation Servlet is based on Sun's JAI API, which contains a wealth of image-format support as well as a robust image-processing engine. Many of the JAI algorithms such as scale, rotate, crop, filter, and resample are especially useful for rendering images in preparation for display.

The Image Adaptation Servlet supports an array of the most popular image interchange formats, including JFIF (JPEG), GIF, BMP, WBMP, and PNG, as well as some of the most popular and efficient image-compression schemes, such as JPEG, GIFLZW, BMPRLE, and DEFLATE. Support for these schemes ensures highly efficient use of the limited bandwidth available for connecting to a wireless device. Finally, the Image Adaptation Servlet supports an array of image content formats, including MONOCHROME, 2BITLUTGRAY, 4BITLUTGRAY, 8BITLUTGRAY, 8BITLUTRGB, 24BITRGB, and 8BITGRAY. This ensures that a diverse array of applications can serve as the source of image content for the wireless device.

In addition to on-the-fly adapting of formats, the Image Adaptation Servlet supports the following scaling and resizing of images:

• Scale to fixed dimensions
• Maintaining aspect ratio, scale to fit in a bounding box
• Reduce size of image data in bytes to honor limits on size placed by device/network

The Image Adaptation Servlet also can intelligently handle various classes of data:

• Categorize input image based on content into MAP, CHART, or PHOTO
• Perform smart scaling and compression for synthetic (computer-generated) images such as MAPS and CHARTS that contain few colors
• Analyze CHART category images to differentially scale axis, labels, and background

Geospatial Rendering

Graphical maps are capable of capturing the complexity of interactions among people, resources, products, and business processes distributed over a geographic space. They are also powerful means for portraying the distribution of attributes such as population density or demographic information that measures income, education, and so forth.

Geographic data has traditionally been managed in proprietary-formatted files and displayed using highly specialized geographic information systems (GIS) applications. With Oracle9i, it is now possible to manage geospatial information natively in the database.

Geospatial data visualization is done by rendering two-dimensional graphical data (points, lines, arcs, polygons, and symbols) into raster graphics format. But there are several additional factors that must be taken into account in map visualization, since maps can use different symbol sets and colors to render the information that lies behind them. Examples are the symbols that will be used to represent various classes of roads, intersections, bridges, and airports. The J2EE platform offers great flexibility in making map visualizations accessible to Java programs, JSP, and wireless applications.

The MapViewer component in Oracle9iAS Release 2 complements the geographic data management capacity of Oracle9i by providing a generic, Web-based means of rendering, delivering, and viewing geographic data in the database.

The MapViewer component is written in Java and runs inside OC4J. MapViewer listens for client requests, which can range from map requests to administrative requests. All requests must be sent using the HTTP POST method, with the content of the requests encoded in XML format.

To obtain a map centered at the San Francisco area (longitude/latitude: -122.40, 37.80), 400 by 360 pixels in width and height, and covering an area of 5.0 decimal degrees, use the following XML request:

<?xml version="1.0" standalone="yes" ?>
<map_request basemap="demo_map"
datasource="mvdemo"
width="400"
height="360">
<center size="5.0">
<geoFeature>
<geometricProperty typeName="center">
<Point>
<coordinates>-122.4, 37.8
</coordinates>
</Point>
</geometricProperty>
</geoFeature>
</center>
</map_request>

Next Steps READ more about Oracle Spatial /products/spatial DOWNLOAD Oracle Spatial code samples /sample_code/products/spatial Oracle9i Jdeveloper /software/products/jdev TAKE an Oracle Spatial class oracle.com/education Course ID: D12848GC20

When a map request is received, MapViewer parses it and retrieves relevant spatial data as well as map metadata (the symbology that defines the map look-and-feel) from the database. A map, which can be visualized in a standard browser, is then rendered and optionally saved to the local file system in a specified format. MapViewer then sends an XML-encoded reply indicating success back to the client.

When issuing a map request to a running instance of MapViewer, the client needs to specify a datasource. A datasource tells MapViewer which database schema to use to retrieve map data and mapping metadata. Datasources can be defined dynamically through administrative requests to MapViewer. For each datasource, MapViewer will establish one or more JDBC connection to the specified database user and will instantiate a specified number of mappers to handle map requests for that datasource.

In MapViewer, a map conceptually consists of one or more themes. Each theme consists of a set of individual geographic features that share certain common attributes. Each feature is rendered and (optionally) labeled with specific styles. Mapping metadata, managed by the Map Definition Tool, controls the appearance of the generated maps. This metadata includes map symbols, text fonts, area and line patterns, styling rules that associate spatial tables with map layers or themes, and base map definitions. Mapping metadata is stored inside the database schema.

Overcoming the Hurdles

Application developers who want to get a foothold in the use of multimedia and geospatial data face several technical hurdles; fortunately Oracle offers a broad range of J2EE technologies that address the development challenges of both JSP and Java Servlet application developers for supporting multimedia and geospatial data.

Joseph Mauro (joseph.mauro@oracle.com) is a principal product manager for Oracle interMedia.