Java Advanced Imaging API

---

General

---

What is the Java Advanced Imaging API?

The Java Advanced Imaging API extends the Java 2 platform by allowing sophisticated, high-performance image processing to be incorporated into Java applets and applications. It is a set of classes providing imaging functionality beyond that of Java 2D and the Java Foundation classes, though it is designed for compatibility with those APIs. This API implements a set of core image processing capabilities including image tiling, regions of interest, deferred execution and a set of core image processing operators, including many common point, area, and frequency domain operators. 

Back to Question Index   

Who needs the Java Advanced Imaging API? 

The Java Advanced Imaging API is intended to meet the needs of technical (medical, seismological, remote sensing, etc.) as well as commercial imaging (such as document production and photography). The API can benefit all Java developers who want to incorporate imaging into their Java applets and applications. 

Back to Question Index   

What features does the Java Advanced Imaging API add to the Java 2 platform? 

• Rich set of functionality for digital imaging. 
• High level of extensibility to allow arbitrary processing capabilities. 
• Support for a wide variety of data types. 
• Deferred Execution. 
• Remote Imaging and truly distributed imaging. 
• Allow multiple implementations with different trade-offs of memory usage, operator optimization, and hardware acceleration. 

Back to Question Index   

Is the Java Advanced Imaging API compatible with the Java 2D API? 

Yes. Applications written using the Java 2D API are completely forward compatible with the Java Advanced Imaging API. 

Back to Question Index   

What other APIs relate to the Java Advanced Imaging API? 

The Java Advanced Imaging API is part of the Java Media API suite that work together to provide customers with enhanced graphics, imaging and communication capabilities. These APIs include: Java 2D for graphics, text, and fundamental image manipulation; Java 3D; Image I/O; the Java Media Framework (JMF) for components to play and control time-based media such as audio and video; Java Shared Data Toolkit; Java Sound; and Java Speech. 

Back to Question Index   

How and when can I get the Java Advanced Imaging API?

A 1.1.2_01 implementation of the Java Advanced Imaging API is available as a standard extension to the Java 2 platform from the Java Advanced Imaging Website.

Back to Question Index   

Is the Java Advanced Imaging API 100% pure Java?

The Java Advanced Imaging API may be run without any native code, i.e., without the code which provides native acceleration. When run in this manner, the Java Advanced Imaging API uses only Java code. However, the Java Advanced Imaging API has not yet been certified as meeting the requirements of the "100% Pure Java" Program per se.

Back to Question Index   

How can I determine whether an operation has a native acceleration method?

The Java Advanced Imaging 1.1.2_01 README lists all the operations that have a native implementation (See section "Operators").

If a native implementation is present it is, by default, the preferred implementation. But if the nature of the sources and parameters of the operation are incompatible with the native operation then processing will revert to Java code. In general the following minimum requirements must be adhered to for the mediaLib native implementation of an operation to be executed:

• All sources must be RenderedImages.
• All sources and destination must have
  • a SampleModel which is a ComponentSampleModel and
  • a ColorModel which is a ComponentColorModel or no ColorModel (i.e., it is null).
• All sources and the destination must have at most 4 bands of pixel data.

Further restrictions may be imposed by individual operations but the above are the most common requirements.

Back to Question Index   

Will the source code for the Java Advanced Imaging API be made available?

The source code for the Java Advanced Imaging (JAI) API is not available at this time. We would however like to hear from customers who might have an interest in obtaining the source code in the future. The exact mechanism by which the source might be made available is at present undefined.

Back to Question Index   

Where can I get more information?

A very good source of code snippets and discussions on more obscure topics in the Java Advanced Imaging API is the jai-interest mail archive, at: http://archives.java.sun.com/archives/jai-interest.html 

The latest API, programmers guide, and other available documents can be found at: http://java.sun.com/products/java-media/jai/ 

Back to Question Index   

How can I unsubscribe from the jai-interest mailing list?

To unsubscribe, send email to listserv@java.sun.com and include in the body of the message signoff jai-interest. For general help, send email to listserv@java.sun.com and include "help" in the body of the message.

Back to Question Index   

Why don't my messages to the jai-interest mailing list appear?

From time to time, in order to prevent unsolicited commercial messages ("spam") from appearing on the list, we may allow postings only from list subscribers. If the email address from which you sent the message does not match your address as it appears in the subscription list, your posting may be disallowed. If this happens to you, please send mail to jai-comments@sun.com   and we will attempt to rectify the situation.

The other situation where it may seem that postings to the jai-interest mailing list do not appear, it may be that the subscription options for you are set to not send you a copy of your posting. To rectify this, send mail to listserv@java.sun.com with the following as body of mail

SET jai-interest@java.sun.com MAIL SUBJECTHDR REPRO NOACK

Back to Question Index   

My question may already be answered in the mail archives - where are they?

The jai-interest archives are available on line at:

http://archives.java.sun.com/archives/jai-interest.html 

The archives are available directly, grouped by month, in reverse chronological order and, within each month, in alphabetical order by subject. Messages appear in the archive almost immediately after being posted to the list.

A search tool is available for the archives. Access to it may be obtained via the page referred to above or directly here.

Before posting a message to the discussion list, please consider searching the archives to determine whether the topic has already been addressed.

Back to Question Index   

Reporting problems to the jai-interest alias.

Please scan this FAQ and the mail archives to see if your problem has already been addressed. If it hasn't, there are several things you can do to increase the chances that we will be able to help you resolve your problem.

Please supply the simplest possible code that exhibits the problem if at all possible. A self-contained, compilable example is best. Where this is not possible, as much context as you can give will be helpful.

For code that generates errors or exceptions, please use the Java command line option "-Djava.compiler=none" as this will produce a stack trace with line numbers. These line numbers are important for us to locate the root cause of most bugs.

Back to Question Index   

Where can I get the Java Advanced Imaging Tutorial?

The Java Advanced Imaging Tutorial is available for download at

http://java.sun.com/products/javamedia/jai  (Click Download)

This tutorial is an interactive Java application that uses the Java Advanced Imaging API to showcase real-world imaging examples, that run as a part of the tutorial. It will help you rapidly create new applications using Java Advanced Imaging.

Back to Question Index   

Does Java Advanced Imaging work with JDK 1.1?

No. You need the Java 2 platform (a/k/a JDK 1.2). Specifically, Java Advanced Imaging 1.1.2_01 requires the Java 2 platform, version 1.3 or higher.

Back to Question Index   

---

Technical

---

How many bands and how many bits per band are supported?

The Java 2D and Java Advanced Imaging APIs support:

• up to 32 bits for integral data
• 32 and 64 bit (float and double) floating-point data
• any number of bands

Users may define their own custom SampleModel classes to support other data types.

Back to Question Index   

What data formats are supported?

The Java Advanced Imaging API supports unsigned byte, signed and unsigned short (Java 2 only partially supports signed short), int, float, and double data types. Complex numbers are represented using two bands, one for the real and one for the imaginary portion of each sample. Certain operators require the use of an integral data type (not float or double), e.g., table lookup and bitwise logical operators. 

Back to Question Index   

What is the best SampleModel to use for performance?

Fastest processing performance will be achieved when the Raster is composed of a PixelInterleavedSampleModel and a DataBuffer type that is supported by native acceleration for the operations of interest. The BandOffsets arrays of all sources and the destinations should match. 

Back to Question Index   

What is the Java Advanced Imaging API doing about performance?

An optional C-based library for Win32, Solaris and Linux provides performance enhancements for many operators. This library is called mediaLib.

Additional hardware optimization on UltraSPARC running Solaris is achieved using the VIS Instruction Set. Additional hardware optimization on the Intel x86 architecture is achieved using MMX.

Back to Question Index   

What is "mediaLib" (or "mlib") in the Java Advanced Imaging API?

mediaLib is a Sun-developed high-performance image processing library. The reference implementation of the Java Advanced Imaging API makes use of a special version of mediaLib which is a subset of mediaLib 2.0 and includes a JNI wrapper. For more information on mediaLib 2.0, see:

http://www.sun.com/sparc/vis/mediaLib.html 

On those Win32 platforms which support MMX, Java Advanced Imaging uses a version of mediaLib containing MMX instructions for acceleration. For those Win32 and Solaris/Intel platforms where MMX is not supported, Java Advanced Imaging uses a pure C version of mediaLib. On Solaris/SPARC platforms, it uses a version containing VIS instructions for acceleration. On the Linux platform, it uses a pure C version of medialib. The use of mediaLib is not essential to the design of the Java Advanced Imaging API; other native platform-specific libraries could be integrated in the future. 

Back to Question Index   

Does Java Advanced Imaging require native code?

No. Native code is supplied for performance purposes only. A C library is supplied for SPARC, Linux and Intel x86 platforms which accelerates most of the operators. In addition there is a lower level SPARC library (VIS) and a MMX library allowing additional hardware acceleration for many operators. If Java Advanced Imaging code does not find these libraries, pure Java code is used.

Back to Question Index   

How can I avoid OutOfMemoryErrors?

There are several possible reasons for an application to produce an OutOfMemoryError.

A common reason is a failure to specify enough memory for the Java virtual machine (JVM) at application startup time. Regardless of how much physical and virtual memory is available on your system, the Java runtime will only allocate a fixed amount, which is determined when the Java Virtual Machine (JVM) starts up. How to allocate more memory to the JVM is discussed below.

JAI creates a TileCache object to store computed portions of images. By default, the cache size is allowed to store up to 16 megabytes of image data. Therefore the JVM should generally be started up with a size larger than this. The actual memory requirements depend on many factors such as the sizes of the images being used and how many operations are used to produce an output image. A setting of -Xmx128m, giving 128 megabytes of (virtual) memory to the JVM is a reasonable starting value. Alternatively, you can specify the size of the tile cache yourself with the TileCache.setMemoryCapacity() method.

Another possible cause of excessive memory use is the use of tile sizes which are too large. By default, images inherit their tile size from their source, i.e. the previous image in a chain. If the tile size is large, more data may be generated than required by an operation. You can specify a smaller tile size for use by an image by creating an ImageLayout object, setting its tile size fields and passing it to the JAI.create() call via a RenderingHints object.

You might also consider adjusting the capacity of the tile cache being used for your particular instance of Java Advanced Imaging. The cache to use for a specific operation may also be specified via the RenderingHints. For all operations, a tile is not calculated until a region of the image overlapping the tile is requested. To save re-computation, the resulting tiles are stored in a cache. When the cache runs out of memory it removes tiles until it has enough memory to store the new tiles. For example, if you are scrolling an image you might scroll to one area and then back to the original area. Tiles required in the original area might have been flushed from the cache in the intervening time, thereby necessitating their recalculation. By allowing recalculation to take place, there is no need to store all of the computed tiles permanently.

The most efficient storage of image data in RAM will be accomplished when the images are tiled and those tiles which are not currently needed are flushed from memory. The deferred execution architecture of the operation chain attempts to process only the tiles that it must. The tile cache attempts to flush those tiles which are no longer needed. These two facilities working in tandem attempt to provide a reasonable tradeoff between memory use and computation.

The operation chain itself does not, however, handle the way image data are managed in the source and sink portions of the chain as this cannot always be controlled by the API itself. For example, if the image source is a file which is not tiled, then the entire image will be loaded. If a tiled source image file is used (e.g., tiled TIFF or FlashPIX), then only those tiles required to execute the operation chain will be read from the disk.

Similarly, at the data sink end of a chain, e.g., the display, if the entire image is requested and held in memory then that will provoke computation of all tiles in all intermediate operations in the chain and consequently reading of the entire image from disk. The data sink will be most efficient if it requests only those tiles that it needs and uses some kind of efficient algorithm for disposing of the destination tiles that it does not need, e.g., those neither being displayed nor adjacent to tiles being displayed.

To reduce RAM usage even further you also have the option of implementing your own tile cache which uses an alternate type of backing storage for the tiles in the cache, e.g., a disk file or a database. Such an approach may lower execution speed in some cases but there are always tradeoffs. There is also a mechanism to replace the "Least Recently Used" algorithm with a custom implementation via the tile cache metric and tile comparator features.

When images with non-standard bit depths (e.g., 1 or 2 bits per sample) are processed, some JAI operators may perform temporary expansion to an 8 bit per sample format. This may require excessive memory in some cases. This behavior was particularly pronounced in the 1.0.2 release; the 1.1 and later releases are significantly more efficient when performing scale, rotate, affine, and transpose operations on 1-bit images.

In JAI 1.0.2, tiles were only removed from the Tile Cache when the cache became full. At this point, tiles were released from memory until 25% of the memory was freed. As of JAI 1.1 and later, tiles can also be released if no "hard references" to the tiles remain. This can occur for example, if an operation goes "out of scope". JAI 1.1 also checks for OutOfMemoryErrors when requesting tiles, and if one occurs, tiles will be released from the cache and the compute request will be reissued. This has significantly reduced OutOfMemoryError conditions in applications. Applications should still check for OutOfMemoryError conditions because other portions of a program can trigger these independent of JAI.

Some improvements were made in JAI 1.1.1 to reduce memory related exceptions while loading image files that may have been noticed with previous JAI versions.

Back to Question Index   

How can I control the amount of memory allocated to the tile cache?

You may obtain a reference to the default tile cache and call its setMemoryCapacity method, supplying the cache capacity in bytes:

TileCache cache =
    JAI.getDefaultInstance().getTileCache();
long size = 32*1024*1024L; // 32 megabytes
cache.setMemoryCapacity(size);

Back to Question Index   

How can I control the amount of memory allocated to the Java runtime?

It may be necessary to set the maximum size of the memory allocation pool of the Java interpreter using the -Xmx command line option. This is particularly important if you are attempting to work with large, untiled imagery.

• -Xmx<size>
  • Sets the maximum size of the memory allocation pool (the garbage collected heap) to <size>. The default is 1 megabyte of memory. The size must be at least 1000 bytes.

    By default, the size is measured in bytes. To specify the size in either kilobytes or megabytes, append "k" for kilobytes or "m" for megabytes.

• -Xms<size>
  • Sets the startup size of the memory allocation pool (the garbage collected heap) to <size>. The default is 4 megabytes of memory. The size must be at least 1000 bytes and must be less than or equal to the maximum memory size (as specified by the -Xmx option).

    By default, the size is measured in bytes. To specify the size in either kilobytes or megabytes, append "k" for kilobytes or "m" for megabytes.

Back to Question Index   

Java Advanced Imaging appears to have a memory leak.

While it is certainly possible that a bug in either the Java runtime or the implementation of the Java Advanced Imaging API itself could cause a memory leak, in most cases what appears to be a memory leak is simply the result of a temporary demand for more memory than is available from the Java runtime. A true memory leak implies that an ever-increasing amount of memory will be required by the application; an application that requests a large amount of memory but will eventually release it may be inefficient but cannot truly be said to exhibit a memory leak. The JAI 1.0.2 implementation of the tile cache allowed the cache to fill completely before releasing tiles, giving the impression of an increasing consumption of memory. JAI 1.1 allows tiles to be released when there are no longer any "hard references" to the operation objects.

Back to Question Index   

What is the best way to display large images efficiently in the Java Advanced Imaging API?

In general, the proper use of tiling is the easiest way to limit the amount of work done. Setting the tile sizes will be very application-dependent. The Java Advanced Imaging API will perform work on a per-tile basis. If the final operator in a chain requires a certain source rectangle to be computed, the actual computation may extend past the actual area needed, up to the next set of tile boundaries.

Experiments have shown that tile sizes of 512x512 and larger appear to be the most efficient. 

Back to Question Index   

When/where are tiles cached in the Java Advanced Imaging API?

The only place where caching takes place automatically is in the method OpImage.getTile. So if you subclass from OpImage and override the computeTile or computeRect methods, but not the getTile method (or you override getTile but call the superclass method somehow), you get caching automatically. Otherwise, you don't. You can construct your own implementations of the TileCache interface or use the factory method in the JAI class and insert tiles manually, or you can construct a NullOpImage around your image which will perform caching.

If you're using an OpImage subclass, but don't want caching, you can call:

this.setTileCache(null);

Back to Question Index   

My image is in color, but the Java Advanced Imaging API is telling me there's only 1 band!

A RenderedImage has a SampleModel and a ColorModel. The SampleModel describes the pixel data the image has and how it is stored in the buffer. The ColorModel interprets the pixel data in a specific ColorSpace. The color definition of a particular band of the pixel is dependent on the ColorModel and its associated ColorSpace. Without a ColorModel, the pixel data of an image has no color definition.

Your image may have an IndexColorModel. In this case, the image data is stored in a 1-banded form, and the ColorModel is used to determine the red, green, blue, and optional alpha values for each pixel. The javax.media.jai.RasterAccessor will automatically cause such images to appear to have 3 or 4 bands as appropriate.

Back to Question Index   

On Solaris, Java Advanced Imaging complains about lack of access to an X server.

Java Advanced Imaging versions previous to JAI 1.1.1 used the AWT toolkit to load GIF and JPEG files. This problem is a manifestation of a JDK bug in which creation of the AWT Toolkit class results in an attempt to open the X display. To work around this problem in Java Advanced Imaging versions prior to 1.1.1, either make an X display available to the Java runtime using the DISPLAY environment variable (no windows will appear on the display), or consider running a dummy X server that will satisfy the AWT, such as the Xvfb utility included with the X11R6.4 distribution.

In the JAI 1.1.1 version, the GIF and JPEG decoders were improved to no longer have a dependency on the X server.

Back to Question Index   

How can I use a BufferedImage as a source for a Java Advanced Imaging operation?

A BufferedImage may be used directly as a source to any Java Advanced Imaging operator or method that calls for a RenderedImage or WritableRenderedImage source or sources.

Back to Question Index   

How can I use a java.awt.Image from the AWT in Java Advanced Imaging?

This code sample reads a GIF or JPEG file using the java.awt.Toolkit class into a java.awt.Image, converts the AWT-Image into a RenderedImage in JAI, then displays the RenderedImage.

You can also do this in Java2D. Use a MediaTracker to ensure the image is loaded, get its width and height and create a BufferedImage with the right dimensions. Then call createGraphics() on the BufferedImage and then call drawImage() on the returned Graphics, passing it the java.awt.Image. Now you have a BufferedImage containing your image data. A BufferedImage is an instance of RenderedImage, so you can pass it to any Java Advanced Imaging interface that calls for a RenderedImage.

An example of creating an image from float data can be found here.

Back to Question Index   

How do I create a PlanarImage from an array of data?

The steps to do this are:

Only the last step involves any actual processing. The rest is just object creation.

Alternatively, a BufferedImage may be constructed directly from the Raster and ColorModel. The RenderedImageAdapter class may then be used to produce a PlanarImage from the BufferedImage. This approach avoids copying entirely.

Sample code to make a RenderedImage out of a 2D array of floats can be found here.

Back to Question Index   

How do I convert images between color spaces?

The Java 2 platform (JDK 1.2 and later) contains a Kodak color management engine which uses ICC profiles for conversion. You will need to provide the ICC profiles in a form from which you can create a java.awt.color.ICC_Profile object. The ICC_Profile can then be used to create a java.awt.color.ICC_Colorspace, which can be used to create a ColorModel that can be supplied to the JAI "colorconvert" operation. If you use the "colorconvert" operation in this manner, and the SampleModel of the source image (and thus the resultant destination image) is not compatible with the ColorModel being specified to the "colorconvert" operation, then you should also provide an appropriate SampleModel in an ImageLayout object passed as a RenderingHint. The SampleModel should be compatible with the ColorModel provided.

ICC profiles for the following color spaces may be downloaded here.

• CMY
• CMYK
• YCbCr based on REC 601
• YCbCr linear based on REC 601
• YCbCr based on REC 701
• YCbCr linear based on REC 709
• Linear Y part of YCbCr for Color Conversion Support

Back to Question Index   

How do I convert any image (with any color model) to a 1 bit image (B&W)?

Use one of the dither operators -- either OrderedDither (fast) or ErrorDiffusion (slower, but higher quality).

First, convert the image to a luminance (grayscale) image and then dither it. You can convert to a grayscale space using the ColorConvert operator:

pb = new ParameterBlock();
pb.addSource(src);
ColorModel cm = 
        new ComponentColorModel(ColorSpace.getInstance(ColorSpace.CS_GRAY),
                                                      new int[] {8}, 
                                                      false, 
                                                      false, 
                                                      Transparency.OPAQUE, 
                                                      DataBuffer.TYPE_BYTE);
pb.add(cm);
PlanarImage dst = JAI.create("ColorConvert", pb);

For simple linear conversions to grayscale, the "bandcombine" operator could also be used.

The dither operation needs to have a lookup table that converts any values less than half the grayscale range to zero and any values above half the range to one. The dither operation will automatically vary the threshold to minimize any contouring effects. Here is some sample code that illustrates using the "errordiffusion" and "ordereddither" operations for dithering a grayscale image to a monochrome (1-bit or bilevel) image.

An alternative to dithering for converting an 8-bit grayscale image to a 1-bit monochrome image is simply to apply a threshold to the image. The "Binarize" operation may be used for this purpose. A threshold may be derived by first creating a Histogram using the "Histogram" operation and then calculating the threshold using one of several methods:

        getIterativeThreshold()
        getMaxEntropyThreshold()
        getMaxVarianceThreshold()
        getMean()
        getMinErrorThreshold()
        getMinFuzzinessThreshold()
        getModeThreshold()
        getPTileThreshold()

This code example demonstrates how "binarize" operation can be used to apply the threshold.

Back to Question Index   

How does the image coordinate system work?

A RenderedImage occupies an arbitrary rectangular area in the 2D plane. Note that, as in the Java 2D API, the Y axis values increase from top to bottom.

Consider this annoted image:

The coordinates of the upper-left corner of the image appears at the point given by the getMinX() and getMinY() methods. The width and height of the image are obtained similarly using the getWidth() and getHeight() methods. In the illustration, the image starts at pixel (-100, -90) and has a width of 330 pixels and height of 215 pixels. The lower right pixel of the image is thus (229, 124). As shown here, the image location is not restricted to the positive quadrant.

Overlayed on the image is its tile grid. Each tile contains a rectangular portion of the image; all tiles have the same width and height and tiles do not overlap. Tiles are referenced by a pair of indices, i.e., tile (2, 3) is to the right and below tile (1, 2). Tiles may extend past the edges of the image. The contents of the portion outside the image are undefined. Only tiles that have some overlap with the image bounds can be obtained by calling getTile. Attempts to retrieve any other tiles will result in a return value of null.

The semantics of an image never depend on the layout of its tile grid layout, but performance maybe affected by the choice of tile size and offsets. The tile grid is defined by the four methods getTileGridXOffset(), getTileGridYOffset(), getTileWidth(), and getTileHeight(). The first two methods yield the position of the upper-left corner of a notional tile (0, 0). The image need not actually contain such a tile. For example, the tile grid of the image illustrated above may renumbered so that the upper left tile has index (-1, -3) and the lower right tile has index (2, -1):

The image doesn't contain a (0, 0) tile at all, and the pixel ( getTileGridXOffset(), getTileGridYOffset()) does not even fall within the image. Both images should act exactly identically when used as a source for all JAI operations except possibly for the output tile grid layout.

The minimum and maximum tile indices may be derived by determining the tile indices of the upper left and lower right pixels of the image. To determine the tile index of a given point, the following methods (available in the PlanarImage class) may be used:

static int XToTileX(int x,
                    int tileGridXOffset,
                    int tileWidth) {
    x -= tileGridXOffset;
    if (x < 0) {
        // Force round to -infinity
        x += 1 - tileWidth;
    }
    return x/tileWidth;
}

static int YToTileY(int y,
                    int tileGridYOffset,
                    int tileHeight) {
    y -= tileGridYOffset;
    if (y < 0) {
        // Force round to -infinity
        y += 1 - tileHeight;
    }
    return y/tileHeight;
}

It is not sufficent to compute (x - tileGridXOffset)/tileWidth since this will produce the wrong result for negative indices. Instead, it is necessary to compute the equivalent of (int)Math.floor((double)(x - tileGridXOffset)/tileWidth). The code above produces the same results using integer arithmetic only.

Most often, images loaded from external sources will have their minimum X and Y and tile grid X and Y offsets all equal to zero. As operations are performed, however, such as scaling, rotation or translation, the coordinates of the resultant images will change.

On a more microscopic level, pixels in a RenderedImage may be thought of as squares with a 1 x 1 extent centered at (x + 0.5, y + 0.5). Geometric mapping operations may need to take this half pixel shift into account.

Unlike the RenderedImage coordinate system, which is discrete, the RenderableImage coordinate system is continuous. The image dimensions are described in terms of floating-point miniumum X and Y coordinates, a height, and an aspect ratio. For the "renderable" operation, you supply the height in the renderable coordinate space and the aspect ratio is derived automatically from the ratio of the source image width to its height (width/height) in the rendered image coordinate system.

When a RenderableImage is rendered, an AffineTransform is applied to map its coordinate system into the rendered pixel coordinate system which is appropriate to a target device such as a display, printer, file, etc. You have to specify the appropriate transformation.

Back to Question Index   

How are regions of interest (ROIs) used?

At this time ROIs are used in Java Advanced Imaging for three purposes:

1. To delimit the area of data to be initialized in the TiledImage set() and setData() methods.
2. To specify the region to be considered in calculating image statistics such as mean, extrema, and histogram.
3. As meta-data which are associated with images via properties.

In the latter case ROIs may be geometrically transformed or "shrunk" by operators in an image chain but otherwise are ignored (this geometric modification of the ROI occurs in parallel with the actual image processing operations). In no case other than the statistics operations does the presence of a ROI affect the operation per se.

One way that you might be able to subtract regions in two images defined by a ROI is as follows:

1. Set up an image chain to calculate the overall difference image.
2. Intersect the two ROIs.
3. Determine the bounding box of the intersection.
4. Create a TiledImage covering the area of the bounding box.
5. Extract from the difference image a Raster of data over the bounding box.
6. Use setData(Raster,ROI) with the Raster of the previous step and the bounding box to set the data of the TiledImage.

Note that the only tiles which will be calculated in the difference image are those which overlap the bounding box of the intersection of the two ROIs. Thus if you are using tiled source and destination images computation will be minimized.

Back to Question Index   

How can I tell when a Java Advanced Imaging operator is complete?

Normally, the deferred execution model makes the notion of "completing" the computation of an image meaningless. Images are computed a tile at a time, and tiles may be computed multiple times if the tile cache is unable to reserve storage for them. However, in some cases such as perfomance benchmarking you may wish to force an image to be computed in its entirety.

JAI.create() is a convenience method which effectively does the following:

1. Validates the sources and parameters of the operations according to the specification of the associated OperationDescriptor.
2. Inserts a RenderedOp node into the imaging chain.

No actual computation of image data will occur until data is requested from a node in the chain unless the isImmediate() method of the associated OperationDescriptor returns true. In this case, the chain will be evaluated as soon as the node is appended to the chain. Otherwise, you can force a chain to be evaluated up to a given node by invoking getRendering() on the RenderedOp of that node. This will force evaluation of the entire chain up to and including that node.

Even at this point, actual tile data will not necessarily have been computed. A loop such as the following may be used to force actual tile computation:

void forceLoad(RenderedImage im) {
    int minX = im.getMinTileX();
    int minY = im.getMinTileY();
    int maxX = minX + im.getNumXTiles();
    int maxY = minY + im.getNumYTiles();

    for (int j = minY; j < maxY; j++) {
        for (int i = minX; i < maxX; i++) {
            im.getTile(i, j);
        }
    }
}

Note that if you wish to measure the execution time of a given node you will need to force computation on its source nodes first to exclude their evaluation time from the measurement. The tile caching should also be disabled.

Back to Question Index   

How do I create an overlay?

As indicated in the specification of the "overlay" operator, it puts one RenderedImage on top of another RenderedImage. To use this operator, two source RenderedImages are needed; call them src1 and src2. Then to create an "overlay" op, you simply do:

RenderedImage dst =
    JAI.create("overlay", src1, src2);

The result dst will have src2 on the top and src1 on the bottom. The part of src1 covered by src2 will not be seen. The image layout (width, height, etc.) of dst is copied from src1.

One constraint is that src1 and src2 must have the same data type and number of bands, or the library will throw an IllegalArgumentException.

The position of src2 is based on its minimum X and Y. If you wish to move src2 to a different location, you must do a "translate" operation on src2 first, and overlay the translated image on src1.

If src2 is bigger than src1, it may cover the entire src1 and you'll only see part of src2 as a result (since dst has the dimensions of src1). To increase the dimensions of an image, the "border" operation may be used.

Back to Question Index   

How do I convert an image with 10 or 12 bit color to 8 bit color?

The most direct approach is to use the "lookup" operator to map your 10 or 12 bit data to 8-bit values. If you just want to do simple range conversion, then, for 12 bit data the lookup table's data can be created by:

byte[] blut = new byte[4096];
for (int i=0; i < 4096; i++) {
    blut[i] = (byte)(i >> 4);
}

A refinement of this is to apply a linear "contrast stretch" (really a brightness adjustment) by mapping the min and max values in the 12 bit image to 0 and 255 in the 8 bit image. (Note that we're cheating here and stuffing values we're interpreting as unsigned into Java's signed byte type).

You can determine the min/max using Extrema operator. You will normally want to choose the min of all bands as the minimum, and likewise for the max. Doing it band by band will cause color shifts. Of course, if the imagery is false color, you may not care.

You then create the table by:

double scale = 255.0/(float)(max-min);
for (int i = min; i <= max; i++) {
   blut[i] = (byte)((i - min)*scale);
}

// Clamp any input values outside
// min/max range, just in case
for (int i = 0; i < min; i++) {
   blut[i] = 0;
}

for (int i = max; i < 4096; i++) {
    blut[i] = (byte)255;
}

You may find that these approaches sacrifice too much detail at the low radiance end of the scale. Frequently a non-linear transformation is needed, usually somewhere between square-root and cube-root.

for (int i = 0; i < 4096; i++) {
    double f = i*(1.0/4095.0);
    // Take the square root
    f = Math.pow(f, 0.5);
    // Scale to 8 bits and round
    blut[i] = (byte)(f*255.0 + 0.5);
}

Back to Question Index   

How do I handle palette-color images?

This type of image, e.g., one derived from a TIFF palette-color file, will have a ColorModel that is an instance of IndexColorModel. In general, if the ColorModel of the source image is an IndexColorModel but that of the destination is not, then the source image is temporarily "expanded" during processing to the number of bands indicated by the IndexColorModel (which is the same as the number returned by OpImage.getExpandedNumBands()).

The destination ColorModel is either specified by an ImageLayout object passed in via the RenderingHints parameter to JAI.create() or it is derived from the SampleModel of the (first) source image: a "best guess" ColorModel is constructed for the destination by passing the source SampleModel to PlanarImage.createColorModel(). If source data are not expanded, it is assumed that the operation in question will apply any special handling needed when it encounters an IndexColorModel case. If it does not specially handle such a case then the raster data will be treated as grayscale (note that if a ColorModel is specified via the RenderingHints it is incumbent on the user to ensure that it will be compatible with the destination SampleModel).

Some operations perform processing on the colormap, effectively changing the colormap and not performing any pixel manipulation. This behavior is accomplished by the operations being subclasses of javax.media.jai.ColormapOpImage. The operations which are implemented as subclasses of ColormapOpImage are "addConst", "andConst", "divideIntoConst", "exp", "invert", "log", "lookup", "multiplyConst", "not", "orConst", "piecewise", "rescale", "subtractFromConst", "threshold" and "xorConst".

As of JAI 1.1.2, a new RenderingHint JAI.KEY_TRANSFORM_ON_COLORMAP specifies whether processing should take place on the colormap, or on the pixel (index) data for those operations that are implemented as subclasses of javax.media.jai.ColormapOpImage. By default, a ColormapOpImage node has a hint with a value of Boolean.TRUE, which means the processing is done on the colormap, and not on the indices. To suppress this behavior, a hint with a value of Boolean.FALSE should be attached to the node.

Other operations, like "translate" (with integral translation factors) and "crop" operations do not actually perform any processing of the data: they effectively forward tile requests on to the source image.

As of JAI 1.1.2, a new RenderingHint JAI.KEY_REPLACE_INDEX_COLOR_MODEL has been provided that allows for automatic color translation for colormapped imagery in those situations where not doing so would result in unexpected / incorrect results (such as geometric operations). Operations that are implemented as subclasses of javax.media.jai.AreaOpImage and javax.media.jai.GeometricOpImage set this RenderingHint to true, such that these operations are performed correctly on the colormapped imagery, not treating the indices into the color map as pixel data. These operations are listed in the The Java Advanced Imaging 1.1.2 README

One of the common uses of the format operator is to cast the pixel values of an image to a given data type. In such a case, since JAI 1.1.2, the format operation adds a RenderingHints object for JAI.KEY_REPLACE_INDEX_COLOR_MODEL with the value of Boolean.TRUE, if the source image provided has an IndexColorModel. Due to the addition of this new RenderingHint, using the "format" operation with source(s) that have an IndexColorModel will cause the destination to have an expanded non- IndexColorModel ColorModel. This expansion ensures that the conversion to a different data type, ColorModel or SampleModel happens correctly such that the indices into the color map (for IndexColorModel images) are not treated as pixel data.

One further thing to note is that even if you pass in an IndexColorModel via the RenderingHints this will not change the way in which the image data per se are handled, although you might be able to display the result. For those operations that are not subclasses of ColormapOpImage, or where color translation is not being performed through the use of JAI.KEY_REPLACE_INDEX_COLOR_MODEL, the image data will still be processed as if they represent grayscale data.

Sometimes it might be useful to convert the palette-color images into RGB images at the beginning of the processing. One way to expand the palette-color images is by inserting a "format" operation which changes the ColorModel, thereby forcing expansion. (Note that as mentioned "crop" and integral translation merely forward the tile requests to the source so you can't expand the data via those operations.) Obviously expanding the palette-color data to RGB will increase the memory footprint but this way you are assured of obtaining a valid result.

Probably the best means of converting a palette-color image to a 3-band RGB image is to use the "lookup" operation. You can construct the lookup table using values returned by the getBlues()/getGreens()/getReds() methods of IndexColorModel; the method getMapSize() should tell you the number of elements in the lookup table. This code example shows how to perform the conversion.

Back to Question Index   

How does Java Advanced Imaging API handle image borders?

Java Advanced Imaging API's "border" operation and BorderExtender classes are capable of extending an image outward for a specified number of pixels in various ways. This is used mainly for geometric operations where interpolation is needed.

To display an image with a decorative border, the Swing toolkit supports several kinds of borders in javax.swing.border package. For some applications, you may want to use an IconJAI (found in the sample directory) to display the image, then add a Swing border to it.

The Java Advanced Imaging API does not have a "drop shadow" or similar special effect operator as found in some imaging software. Such an effect could be created by overlaying an image over a modified version of itself to produce a 3D look.

Back to Question Index   

How do I convert an IndexColorModel into a grayscale image?

A method to convert an CS_sRGB into a grayscale Image (CS_GRAY) can be found here.

Back to Question Index   

How can I perform convolution with a different kernel for each band?

If you intend to convolve all bands of a given multi-band image (the common case) with the same kernel then you can use the "convolve" operation as is. If however you would like to convolve each band of a multi-band image with a different kernel then you would need to extract each band, e.g., using "bandselect", apply the appropriate convolution kernel to each band using "convolve", and then merge the separate filtered bands into a resultant multi-band image.

If you are attempting to do some sort of multispectral template matching however, the above scenario might not be what you want. In that case, you could use single-band convolution by, for example, first converting the images to a different color space such as HIS or YCC and perform the template matching using the intensity or luminance band.

Back to Question Index   

Can I serialize a TiledImage?

A TiledImage may not be serialized directly as it does not implement the java.io.Serializable interface. You may however construct a SerializableRenderedImage from a TiledImage. See the javax.media.jai.remote.SerializableRenderedImage documentation for more information. Alternatively, if a suitable external file format exists the ImageCodec interfaces may be useful.

Back to Question Index   

How can I create and run an Applet?

In order to use Java Advanced Imaging in a browser environment, your applet HTML file should be converted with the Java Plug-in HTMLConverter (now sits in the JDK 1.3.1 and JDK 1.4 bin directories). The Java Plug-in (now part of JRE/JDK 1.3.1 and 1.4) and JAI configuration requirements are as follows:

1. Install the Java Plugin for your architecture (Solaris, Windows, etc.). The Java Plug-in is available from the Java Plug-in download page. The Plug-in is now part of the JRE. Follow the links to download the plugins and patches for the platform of your choice. There are also pointers to a FAQ, Documentation and other relevant information at this site.

2. Follow the Java Plug-in Browser Registration Instructions for the platform of your choice
   • Solaris Browser Registration Instructions
   • Linux Browser Registration Instructions
   • Windows Browser Registration Instructions. Look for the section on running Netscape 7.x/Mozilla 1.x with Java Plug-in.

3. Once the Java Plug-in is registered with the browser, download the JAI JRE installation executable
   • On Solaris: jai-1_1_2_01-lib-solaris-sparc-jre.bin
   • On Linux: jai-1_1_2_01-lib-linux-i586-jre.bin
   • On Windows: jai-1_1_2_01-lib-windows-i586-jre.exe

4. Follow the installation instruction to install JRE version of JAI into the Java Plugin just installed.

5. After JAI is installed in the manner described above, certain permissions may need to be added to the

   • Java policy file

     $JREHOME/lib/security/java.policy
     

   • Or user java policy file
     On Solaris:

     $HOME/.java.policy

     On Windows (NT, for example)

     C:\WINNT\Profiles\$User\.java.policy

   The permissions needed to be added are:

   • On solaris:

     grant {
       // Necessary for JAI to utilize mediaLib in applets.
       permission java.io.FilePermission "/usr/bin/uname", "execute";
       // May be necessary for JAI to do encoding in applets.
       permission java.io.FilePermission "${java.io.tmpdir}/*", "write, delete";
     };
     

   • On Windows:

     grant {
       // May be necessary for JAI to do encoding in applets.
       permission java.io.FilePermission "${java.io.tmpdir}/*", "write, delete";
     };
     

   If the users encode images in the applet, the applet may write to some temporary file. If the users don't encode images in the applet, no permission is needed on Windows. As mentioned above, on Solaris, the user still needs to grant execute permission to /usr/bin/uname on Solaris.

             

    public static void init() {
        OperationRegistry    or =
                JAI.getDefaultInstance().getOperationRegistry();

        // An OperationDescriptor describing the "fourier" transform
        // operator
        OperationDescriptor  fd = new FourierDescriptor();

        // A RenderedImageFactory which could presumably create
        // FourierOpImage's
        RenderedImageFactory rifJava = new FourierRIF();

        // A RenderedImageFactory which could create a
        // NativeFourierOpImage which used a native implementation
        // of the fourier transform.
        RenderedImageFactory rifNative = new NativeFourierRIF();

        // First register the "fourier" OperationDescriptor
        or.registerDescriptor(fd);

        // Now register the two RenderedImageFactory-s for this operator
        RIFRegistry.register(or, fd.getName(), "com.xxx.yyy", rifJava);
        RIFRegistry.register(or, fd.getName(), "com.xxx.yyy", rifNative);

        // Now set a preference such that the Native implementation
        // is preferred over the java implementation
        RIFRegistry.setPreference(or, fd.getName(),
                            "com.xxx.yyy", rifNative, rifJava);
    }

          

String fileName; // Path of the file to be read
PlanarImage image =
    JAI.create("fileload", fileName);

URL url; // URL of image to be read
PlanarImage pi = JAI.create("URL", url);
BufferedImage bi = pi.getAsBufferedImage();

RenderedOp op = JAI.create("filestore", image, filename, filetype, encodeParam);

// Load the input image.
String inputFile = "image.bmp";
RenderedOp src = JAI.create("fileload", inputFile);

// Store it as a JPEG
String outputFile = "image.jpg";
JAI.create("filestore", src, outputFile, "jpeg",
           (ImageEncodeParam)null);

ParameterBlock pb = new ParameterBlock();
pb.add(tAWTImage);
PlanarImage tPlanarImage =
    (PlanarImage) JAI.create("awtImage", pb );

OutputStream tOutput =
    new FileOutputStream(tFilename);
ImageEncoder tEncoder =
    ImageCodec.createImageEncoder("BMP",
                                  tOutput,
                                  null);
tEncoder.encode(tPlanarImage);
tOutput.close();

RenderedImage bg; // background image
RenderedImage fg; // foreground (inset) image

TiledImage ti =
   new TiledImage(bg.getMinX(),
                  bg.getMinY(),
                  bg.getWidth(),
                  bg.getHeight(),
                  bg.getTileGridXOffset(),
                  bg.getTileGridYOffset(),
                  bg.getSampleModel(),
                  bg.getColorModel());
ti.set(bg);

Rectangle r =
    new Rectangle(fg.getMinX(),
                  fg.getMinY(),
                  fg.getWidth(),
                  fg.getHeight());
ROI roi = new ROIShape(r);
ti.setData(fg.getData(), roi);

public void display(Collection imgs) {
    Frame window = new Frame(testName);
    int n = imgs.size();
    window.setLayout(new GridLayout(1, n));

    Iterator iter = imgs.iterator();
    while (iter.hasNext()) {
        RenderedImage img =
            (RenderedImage)iter.next();
        int w = img.getWidth();
        int h = img.getHeight();
        ScrollingImagePanel panel =
            new ScrollingImagePanel(img, w, h);
        window.add(panel);
    }

    window.pack();
    window.show();
}